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Abstract 


This report documents a case study on the application of 
Reliability Engineering techniques to achieve an optimal balance 
between performance and robustness by tuning the functional 
parameters of a complex non-linear control system. For complex 
systems with intricate and non-linear patterns of interaction 
between system components, analytical derivation of a 
mathematical model of system performance and robustness in 
terms of functional parameters may not be feasible or cost- 
effective. The demonstrated approach is simple, structured, 
effective, repeatable, and cost and time efficient. This general 
approach is suitable for a wide range of systems. 
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1. Introduction 


This report documents a case study on the application of Reliability Engineering techniques to achieve 
an optimal balance between performance and robustness by tuning the functional parameters of a complex 
non-linear control system. The system of interest is the nonlinear pendulum control system reported by 
Torres-Pomales and Gonzalez [2]. The following subsection examines fundamental systems concepts and 
establishes a general context for the case study. 

1.1. Background Systems Concepts 

An engineered system is an entity with a purpose that is achieved by the interactions (i.e., relations) of 
its internal components (i.e., sub-entities) with each other and with the external environment (i.e., 
surroundings). The concept of system is circular and recursive such that, in general, both the environment 
and the components of a system are themselves systems. A system of interest (SOI) (or system in focus) is 
always contained within a larger super-system (or meta-system) consisting of the SOI and its environment. 

The purpose (i.e., a goal) of an engineered system is defined in terms of the desired (i.e., intended) 
effect on its environment achieved by the flow of matter, energy, or information between the system and 
its environment. The system purpose is the basis for the specification of the desired functional and quality 
attributes (i.e., properties) of the system. The purpose of system components is to contribute to 
accomplishing the larger goal of the containing system. Thus, in general, there is a dependence (i.e., a 
relation) between a system and its environment. Avizienis et al. define system dependability as “the ability 
to deliver service that can justifiably be trusted” and also “the ability to avoid service failures that are more 
frequent and more severe than is acceptable” [1]. Dependability is a function of criticality, which is a 
measure of the importance (i.e., degree or strength of influence) of a factor in achieving a larger goal. The 
failure or variability in performance of a high criticality component can have a severe effect on the operation 
of the containing system. Dependability is also a function of trust (i.e., confidence) in the service delivered 
by a system. Assurance is the level of confidence in the properties of a system. This confidence is bounded 
by the various forms of uncertainty (both epistemic and aleatoric) that threaten the ability to achieve and 
ascertain the desired system properties. Uncertainties can originate internal to the system, or they can be 
external in nature and their effects propagate to the system (for example, at the inputs). Dependability is 
bounded by the ability to measure uncertainties and predict their effects on goals. In general, the ability to 
assess uncertainties and their effects diminishes as system complexity (i.e., number and variety of 
components, relations, and states) increases. The dependability of a component (or sub-system) can be 
seen as a measure of the level of associated risk (i.e., potential for unwanted outcome) with respect to the 
goals of the containing system. 

Goals, dependencies, and uncertainties are interrelated. In engineering a system, we want to achieve a 
balance between performance (i.e., utility or value with respect to purpose) and robustness (i.e., 
insensitivity to uncertainties and undesired influences) in achieving the goals. The relation between goals 
and uncertainties is primarily determined by the dependencies defined in the system architecture and is a 
function of system parameters. Once a system architecture has been specified, we want to determine the 
combination of parameter values that results in the optimal balance between performance and robustness. 
For complex systems with intricate and non-linear patterns of interaction between system components, 
analytical derivation of a mathematical model of system performance and robustness in terms of functional 
parameters may not be feasible or cost-effective. Even if a model can be derived, closed-form solutions 
may not be possible. Without effective analytical means to determine the best combination of system 
parameters, the alternative option is experimentation. However, trial-and-error can be time consuming and 
the results can be unpredictable. We want a methodical, effective, repeatable, and cost and time efficient 
approach to tune the parameters of a system. 
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1.2. Overview of Content 


The next section describes the system of interest, including the plant and the controller. This is followed 
by the system tuning problem statement. An overview of the Reliability Engineering concepts applied in 
this study is given. This is followed by a description of the approach used to tune the system. Detailed 
results are presented. Conclusions and final remarks end the report. 


2. System Under Study 

Figure 1 is the block diagram of the control system. The state of the plant X is controlled by the motor 
torque Td. The system is designed to track the target arm position 0d while maintaining control of the 
pendulum above the horizontal. The swing-up problem was not part of this study. 



2.1. Plant 

The plant consists of a motor which generates a torque x, a vertical shaft, a horizontal arm with position 
and speed denoted 0 and 0, and a freely rotating pendulum with position and speed denoted § and 0. For 
this case study, it is assumed that the pendulum is always above the horizontal position, i.e., § < ± 90°. Two 
plant models were developed. The Ideal Plant Model does not include motor dynamics and assumes that 
the desired motor torque equals the applied torque (i.e., Td = t). The Laboratory Plant Model includes 
model dynamics, friction, and other hard nonlinearities. 

2.1.1. Ideal Plant Model 

The ideal plant model was derived using the Euler-Lagrange equation. Variable m denotes the 
pendulum mass, Ji is the moment of inertia of the shaft and arm relative to the shaft’s main axis, J 2 denotes 
the moment of inertia of the pendulum with respect to its center of mass, Li is the length of the arm, L 2 is 
the length of the pendulum, and g is the gravitational constant. 
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Ad + B (p + C — t 


( 1 ) 


D0 + E0 + F + G = 0 (2) 

where: 

A = Ji + mLi 2 + (!4) mL 2 2 sin 2 ((|)) E = J 2 + (%) 111 L 2 2 

B = (V 2 ) mLiL 2 cos(^) F = - (V 8 ) mL 2 2 sin(2^)(0) 2 

C = - (V 2 ) mLiL2sin((l))(0) 2 + (!4) mL2 2 sin(2(|))(00) G = - (!4) mgL 2 sin((|)) 

D = B 

2.1.2. Laboratory Plant Model 

The laboratory plant model was developed starting from the ideal model and adds motor dynamics, 
friction, and some hard nonlinearities. 


A e 9 + B e 0 + C e — Td 


( 3 ) 

D e 9 + E e 0 + F e + G e + H e — 0 


(4) 

where: 



Ae — A H - J m 

Ee = E 


B e = B 

F e = F 


C e — C + a m 9 

G e = G 


D e = D 

He = 0/2) bp <j> 



Here Td denotes the torque developed by the motor, J m denotes the rotational inertia of the motor, and 
a m and b p are viscous friction terms. In addition, the Laboratory Plant Model includes torque saturation 
Td,max, motor angular speed saturation 0 d ,max> static and dynamic motor friction T s tatic and Tdynamic, and input 
torque dead zone Td,dead. In addition, scaling factors were used to more closely match the response of the 
model to the physical plant. Table 1 lists the model parameter values. 
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Table 1: Parameter Values for Laboratory Plant Model 


Parameter 

Value 

Parameter 

Value 

Parameter 

Value 

Ji 

0.04699 kg-m2 

am 

0.01932 N-m-s 

dAe 

1.20 

h 

0.00780 kg-m2 

bp 

0.01 N-m-s 

dBe 

1.12 

J m 

0.03790 kg-m2 

T d,max 

9.8 N-m 

dCe 

1.05 

Li 

0.36 m 

@d,max 

28.27 rad/s 

dDe 

1.12 

U 

0.28 m 

T static 

0.10 N-m 

dE e 

0.98 

m 

1.1564 kg 

T dynamic 

0.08 N-m 

dFe 

1.00 

g 

9.8 m/s2 

T d,dead 

0.147 N-m 

dGe 

1.08 


dH e 

5.50 


2.2. Controller 

Figure 2 illustrates the structure of the system controller. This is a dual-loop architecture in which the 
inner loop is responsible for the state of the pendulum and the outer loop generates pendulum position 
commands based on the target arm trajectory. 


Target 



Figure 2: Block Diagram of System Controller 


The system control laws were developed based on a Simplified Plant Model with scaling factors added 
to minimize the modeling error relative to the Laboratory Plant Model. The equations of the Simplified 
Plant Model are: 

0=H a +L a T d (5) 

0 = Q a + R a T d (6) 

where: 
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H a = -[A a (F a + Ga)]/(A a E a - B a D a ) 

O 

p 

II 

o 

L a = -Da/(A a Ea - B a D a ) 

Da = D 

Q a = Ba(F a + Ga)]/(A a E a - B a D a ) 

E a = h + (%) mL 2 2 

R a = E a /(A a Ea - BaDa) 

Fa = - (V 8 ) mL 2 2 (0) 2 

A a = Ji + mLi 2 

Ga = - CA) mgL 2 (<}>) 

Ba = ( l A) mLiL 2 



The scaling factors in Table 2 were applied to the Simplified Plant Model. 

Table 2: Scaling Factors for Simplified Plant Model 


Parameter 

Value 

Parameter 

Value 

dAa 

1.35 

dEa 

0.98 

dBa 

1.00 

dFa 

1.05 

dCa 

0.00 

dGa 

1.00 

dDa 

1.00 

dLa 

1.05 


2.2.1. Pendulum Controller 


The pendulum control law used the sliding controller theory presented by Slotine and Li [2]. The 
sliding surface is: 


s= (s +/i ) (/„ 


(p dt ) — (p + 2\<p + I (pdt 


J o 


where 0 = (p — (p d . The desired motor torque is: 

T d = (- H a + V + W)/L a 

with: 


( 7 ) 


V = 4 > d - 2A0 - A 2 0 

CO 

II 

e 

W = -K sat(s/0) 

P = J W/Li 
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K = P[F+r|] + (p-l)|u| 
u = — H a + V 


F — |H U - Hi | 


Factors H u , H 1? L u , and are upper (u) and lower (1) bounds for H and L of the Laboratory Plant Model. 
These factors are given by the corresponding factors in the Simplified Plant Model using the scaling factors 
in Table 3. The tuning parameters for the Pendulum Controller are X, 8, and r|. 


Table 3: Scaling Factors for Error Bound Models 


Parameter 

Value 

Parameter 

Value 

dA u 

1.25 

dAi 

1.60 

dB u 

1.00 

dB, 

1.00 

dC u 

0.00 

dCi 

0.00 

dD u 

1.00 

dD, 

1.00 

dE u 

0.90 

dEi 

0.98 

dF u 

1.30 

dF, 

0.85 

dG u 

1.30 

dGi 

0.80 

dL u 

0.90 

dL, 

0.90 


2.2.2. Arm Controller 

The arm control law is a PID law (Proportional, Integral, Derivative) with a nonlinear scaling factor to 
map the desired arm angular acceleration to the desired pendulum position. 

0d = [— K,£ - K 2 e - K 3 J 0 f e dt]/ [Q) (L x /L 2 )0 2 + (g/L,)] (8) 

where 6 = 6 — 6 d . The tuning parameters for the Arm Controller are Ki, K2, and K3. 


3. System Tuning Problem 

The purpose of the control system is to move the arm following the trajectory given by the Target Arm 
Position 0d while balancing the pendulum. Ideally, the arm would perfectly track the target position with 
no effects due to variations in the characteristics of the plant. In real life, it is not possible to mitigate 
completely the effects of delays and functional coupling between system elements. Thus, the goal is to 
achieve an optimal balance between arm tracking performance and robustness. The robustness attribute 
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would account for plant modeling errors and variability in the characteristics of the physical plant. Note 
that swing-up of the pendulum is not within the scope of this arm-tracking problem. The applicable system 
initial conditions include those where the pendulum is above the horizontal position < ±90°) and the 
combined values of the plant state variables (| ), 0, 0, and 6 are within the stable region of the control system. 
It is also assumed that deriving a closed-form analytical derivation of optimal parameter settings is not 
feasible. 


4. Reliability Engineering Concepts 

Statistical techniques from the field of Reliability Engineering enable learning about and 
characterization of products and processes by means of systematic experimentation and modeling. The 
following sections present brief overviews of these topics. 

4.1. Design of Experiments 

Design of Experiments (DOE) theory is applied in the development of strategies to investigate the 
effects of independent controlled variables on system performance measured at selected variables of interest 
[4] [5]. The system of interest (SOI) is modeled as a black box to study the relations between Control, 
Input Signal, and Noise Factors (i.e., variables, parameters) and the Response Variable, as illustrated in 
Figure 3. The Noise Factors represent sources of system variability. In this model, the value of the response 
variable is a function of the Control, Input Signal, and Noise variables: y = f(X, S, Z). We seek efficient 
experiments that yield insightful knowledge about the system with as few resources as possible. 


Control 
Factors X 


Input 

Signal 

Factors 



Response 
Variable y 


Noise 
Factors Z 


Figure 3: High-Level Black-box Model of the System of Interest (SOI) 


In a basic experiment, we study the relation between the Control Factors (denoted xi, X 2 , Xk) and 
the Response Variable (denoted y) while the Input Signal Factors are given and the Noise Factors are not 
controlled. An experiment consists of a series of test points in which the Control Factors are assigned 
particular values and the Response is measured. In a two-level experiment, the Control Factors are tested 
at two different values. For a quantitative variable, the levels are the extremes of the value range, denoted 
Xmax and Xmin. For a qualitative variable, which has discrete values, the two levels are different values in the 
variable’s value set. A Full Factorial (Experiment) Design consists of all possible combinations of the 
control variables. This is a complete and systematic study of the design space defined by the Control 
Factors. A two-level full factorial design consists of 2 k test points. A Fractional Factorial Design includes 
a subset of test points in a Full Factorial Design. There are 2 k " p test points in a two-level fractional factorial 
design, which consists with k-p independently assigned control factors and p confounded factors that are 
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assigned values as function of the independent control factors. 

In designing experiments, there is a tradeoff between the quality of data and the cost needed to generate 
it. In general, we want to maximize the accuracy of the characterization made possible by the data (i.e., the 
benefit) while minimizing the number of test points (i.e., the cost). The simplest way to increase the quality 
of information generated by an experiment is to increase the number of levels of the control factors. A 
three-level fractional factorial design consists of 3 k " p test points. The obvious problem with this approach 
is that the number of test points (and thus, the cost of the experiment) increases exponentially. A Central 
Composite Design (CCD) consisting of full or factorial experiment portion, a center portion, and an axial 
portion offers a more favorable tradeoff between quality and size of an experiment [6]. Figure 4 illustrates 
the structure of a two-factor CCD experiment. The values -1 and +1 are standardized representations of the 
variables values, x m in and x ma x, respectively. In this figure, the four points at the corners of the box are the 
factorial portion of the experiment: (±1, ±1). Point (0, 0) is the central portion. The points at the ends 
of the central axes (0, ±a) and (±a, 0) are the axial portion. Notice that this is five-level experiment with 
the control factors taking values 0, ±1, and ±a. When a = 1, the experiment is called a Face-Centered 
Design and the factors take only three possible values: 0 and ±1. The number of test points in a CCD 
experiment is 2 k " p + 2k + 1 . 


A 


x 2 


-i, +1L 


(0, +a) 

0 

1 
I 


(o, o) 


(-a, 0) O H ^ _ 


(+ 1 , + 1 ) 


-4>(+a, 0) 


- 1 , - 1 ) 


__l 

I 

6 

(0, -a) 


6 


< > 

Xi 

Figure 4: Structure of a Central Composite Design (CCD) 


4.2. Response Modeling 

We want to use the results of an experiment to generate a mathematical model of the relation between 
the control factors x and the response variable y. Here, we consider only polynomial models. A two-level 
experiment that tests each factor only at two values contains enough information to build a linear model. 
In addition to the direct relation between each control factor and the response variable, we want to consider 
possible interactions between control factors. The general expression for a linear polynomial model with 
two-factor interaction terms is: 

y = b 0 + ^ biXi + SiSj bijXiXj (9) 

where 1 < i < k, 1 < j < k, and i ^ j. If the variables are assigned values in the ±1 as in Figure 4, the 
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magnitude of the coefficients bi is a measure of the strength of the effect (and hence, importance) of the 
corresponding factor Xi or interaction XiXj. Note that higher-order interactions are possible, but as a rule of 
thumb, the strength of interactions decreases as their order (i.e., number of interacting factors) increases. 
For a three-level experiment such as a CCD, the system response can be modeled with a second-order 
polynomial. 

y = b 0 + Si biXi + SiSj, i*j bijXiXj + Si biiXi 2 (10) 

This model can achieve better accuracy for systems with non-linear responses. 

The basic relation between modeling and experiment design is that high-order models can achieve 
better accuracy but require experiments with higher granularity and, thus, higher cost. We need models 
that have adequate fidelity for the intended use at a reasonable cost. Analysis of the application and careful 
judgment are required to determine the best combination of experiment design and response model. 

4.3. Optimal and Robust Parameter Design 

Once a model has been developed, an optimization tool can be used to determine the combination of 
control factors that results in the best system response. There are three categories of optimization problems: 
the-smaller-the-better, the-larger-the-better, and the-closer-to-nominal-the -better (i.e., as close as possible 
to a target). The quality of system performance is determined not only on achieving the desired target value 
under ideal conditions, but must also take into accounts the effects of variability in uncontrolled noise 
factors. To achieve a high quality system design, both the mean and the dispersion in system performance 
must be controlled. An optimal parameter design achieves the best balance between performance and 
robustness (i.e., insensitivity to variability). For this, we need a parameter optimization approach that can 
account for the mean value y and deviation a y of the system response. One approach to this is the use of 
Dual Response Surfaces, in which models are developed for the mean response and the standard deviation, 
and an optimizer is used to minimize the deviation with a constraint on the error of the mean response 
relative to the desired target [7]. Another approach described below is to optimize signal-to-noise ratios, 
which is the approach used in the case study reported here. 

To enable a robust parameter design, the experiment must generate data suitable for the characterization 
of the effects of the noise factors z (see Figure 3). Note that the categorization of factors as either control 
or noise can be a matter of judgment based on the goals of the parameter design activity. Factors with weak 
effects on the response variable could be classified as noise in order to simplify the design activity by 
focusing on the strongest control factors. Because of this, it is possible that some noise factors are 
controllable in an experiment and others may really be uncontrollable. For simulation-based experiments 
such as the one considered in the case study reported here, the set of controllable noise factors may be larger 
than for real physical experiments. Figure 5 illustrates the high-level structure of an experiment intended 
to characterization and optimization of system performance and robustness. The inner and outer arrays are 
independent experiments for control and noise factors as described in preceding sections. The complete 
outer array experiment is executed for each test point of control factors in the inner array. A response 
measure is made for every test point of control and noise factors, and robustness measures (such as mean 
response and deviation) are made by combining the results of a complete noise factors experiment for a 
particular control factors test point. 

The signal-to-noise ratio is a robustness measure intended to capture with a single metric both the 
deviation of the average response from the target response y m and the dispersion of the response. The mean 
square deviation of the response is given by: 
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( 11 ) 


Amsd (y Ym) CTy 

The signal -to-noise ratios for the optimization problems of the-smaller-the-better, the-larger-the -better, 
and the-closer-to-nominal-the-better are given in Table 4. This robustness measure is computed for each 
test point of the inner array in Figure 5. At the completion of the experiment, a model can be built to relate 
the control factors to the signal -to-noise ratio SN. A control factor optimization based on the maximization 
of the signal-to-noise ratio gives the system configuration with the best balance of performance and 
robustness. 


12 3 ... r 



Outer 

Experiment 

Array 


Inner 

Experiment 

Array 

Response 

Measures 

Array 

Robustness 

Measures 

Array 


Figure 5: Structure of Robust Parameter Design Experiment 


Table 4: Formulas for Signal-to-Noise Ratio 


Case 

Signal-to-Noise Ratio (SN) 

the-smaller-the-better 

-10 logio[(l/r) Si yi 2 ] 

the-larger-the -better 

-10 logio[(l/r) Si (1/yi 2 )] 

the-closer-to-nominal-the-better 

10 logio[y 2 / cry 2 ] 


5. System Tuning Approach 

This section describes the elements of the approach followed to tune the parameters of the control 
system. This case study was intended as a proof of concept and demonstration, and as such, the tuning 
problem was scoped to be representative but without excessive complexity. MATLAB was used to model 
the system and perform simulation experiments. 
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5.1. Control and Noise Factors 


Table 5 lists the chosen control and noise factors. The baseline values for the noise factors are the 
nominal values given in Table 1 . The baseline values for the Arm Controller and the Pendulum Controller 
factors where determined by trial-and-error to result in a stable system (i.e., small state perturbations are 
mitigated asymptotically) with the region of convergence for the pendulum state given in Table 6 (i.e., if 
the initial pendulum state is in this region, the plant state will converge the zero state (c|), 0, 0, and 6 ) = (0, 
0, 0, 0), assuming 0d = 0. The ranges of the control and noise factors were specified as percentages of the 
baselines values. 


Table 5: Control and Noise Factors for the Control System 


Category 

System 

Element 

Factor 

Baseline 

Value 

Range 

(%) 

Control 

Factors 

Arm 

Controller 

Ki 

2.50 

±50 

k 2 

2.00 

±50 

k 3 

0.10 

±50 

Pendulum 

Controller 

X 

20.00 

±50 

8 

0.017 

±50 

Noise 

Factors 

Laboratory 
Plant Model 

Li 

0.36 

±15 

U 

0.28 

±15 

m 

1.1564 

±15 


Table 6: Region of Convergence of Baseline Control System 


Initial 
Pendulum 
Position <|) (°) 

Range of 

Initial 

Pendulum 

Speeds 0 (°/s) 

Initial 
Pendulum 
Position <|) (°) 

Range of 

Initial 

Pendulum 

Speeds 0 (°/s) 

+35 

(-600, +30) 

-35 

(-30, +600) 

+25 

(-500, +190) 

-25 

(-190, +500) 

+15 

(-525, +340) 

-15 

(-340, +525) 

+5 

(-525, +500) 

-5 

(-500, +525) 
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5.2. Stimulus Tracking Signal 


The chosen Arm Position Target 0d, which corresponds to the Input Signal in Figure 3, was a ramp 
waveform from 0° to 90° with a 307s slope from seconds 2 to 5. This is illustrated in Figure 6. 


Arm Position Target 



5.3. Response Metrics 

Each experimental test point consisted of a 30 second run in which the control and noise factors were 
held constant and the stimulus signal was applied. Each run was divided into two phases: a transient phase 
(denoted 1) from 0 to 15 seconds, and a steady-state phase (denoted u) from 15 to 30 seconds. The goal of 
the control system is to minimize the arm tracking error, defined as e = 9 = 0 - 0d, while balancing the 
pendulum. The performance metrics are given in Table 7, where Ti = 15 seconds and T u = 30 seconds. 
Note that an outer array experiment consists of m test points (see Figure 5), and the signal-to-noise ratio is 
computed based on the results of one outer array experiment (i.e., one inner array row). 
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Table 7: System Response Performance Metrics 


Experiment 

Segment 

Phase 

Description 

Label 

Formula 

Test Point (i, j) 

Transient 

Mean Square 
Error (MSE) 

Pi 


Maximum Square 
Error 

ei 2 

max(e 2 ) for 0 < t < Ti 

Steady 

State 

Mean Square 
Error (MSE) 

Pu 

LMfVd, 

\T U - T j/ J Tj 

Maximum Square 
Error 

e u 2 

max(e 2 ) for Ti < t < T u 

All 

Combined Error 

E 

wiPi + W2ei 2 + W3P11 + W4e u 2 ; w q = 0.25 

Inner Array 
Row 


Signal-to-Noise 

Ratio 

SN 

— [&i>] 


5.4. Experiment Design 

The Inner Experiment Array for the Control Factors was a Central Composite Design for the five factors 
Ki, K2, K3, X, and c, which are denoted Xi, X2, X3, X4, and X5. The factorial portion of the Inner Array was 
a half-fractional factorial design with confounded factor X5 = X1X2X3X4. 

The Outer Experiment Array for the Noise Factors was a half-fractional factorial design for the three 
factors Li, L2, and m denoted Ni, N2, and N3. The confounded noise factor was N3 = N1N2. 


6. Results 

As stated above, the experiment was performed using MATLAB. This section describes the experiment 
results and the parameter design with optimal signal-to-noise ratio. 

6.1. Experimental Response Measures 

Table 8 shows the results of the experiment. The combined error measures for each test point are given 
in the columns labeled El to E 4 . The signal-to-noise ratio for each row of control factors is given in the 
S/N column. 

6.2. Response Model 

The system response was modeled with a quadratic polynomial as in equation ( 10 ). Rather than 
including all the two-factor terms, an effects analysis was performed to identify the strongest interactions. 
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These were X1X2, X1X3, X1X4, X2X3, and X1X5. The full model for the signal-to-noise ratio is: 

SN = b 0 + biXi + b 2 X 2 + b 3 X 3 + b 4 X 4 + b 5 X 5 + bi 2 XiX 2 + bi 3 XiX 3 + bi 4 XiX 4 + b 23 X 2 X 3 + bi 5 XiX 5 + bnXi 2 
+ b 22 X 2 2 + b 33 X 3 3 + b 44 X 4 4 + b 55 X 5 5 

The model coefficients were determined using regression. The values are given in Table 9 


Table 8: Experiment System Response Measures 


Noise Factors 



Control Factors 




Nl/Ll 

N2 / L2 

N3/m 

-1 

-1 

1 

1 

-1 

-1 

-1 

1 

-1 

1 

1 

1 



Xl/kl X2 / k2 

X3 / k3 

X4/lambda 


X5/epsilon 


El E2 

E3 

E4 


S/N 

1 

-1 

-1 

-1 

-1 


1 

164.759 

211.915 

169.640 

202.773 

- 104.769 

2 

1 

-1 

-1 

-1 


-1 

92.331 

105.338 

98.999 

101.663 

- 92.042 

3 

-1 

1 

-1 

-1 


-1 

140.695 

174.424 

142.577 

172.215 

- 101.287 

4 

1 

1 

-1 

-1 


1 

101.038 

102.994 

138.310 

98.582 

- 94.268 

5 

-1 

-1 

1 

-1 


-1 

164.944 

240.843 

168.811 

230.420 

- 106.383 

6 

1 

-1 

1 

-1 


1 

96.915 

105.989 

107.866 

101.518 

- 92.725 

7 

-1 

1 

1 

-1 


1 

145.174 

171.730 

152.304 

162.395 

- 101.280 

8 

1 

1 

1 

-1 


-1 

91.764 

103.789 

125.609 

99.935 

- 93.272 

9 

-1 

-1 

-1 

1 


-1 

155.422 

203.201 

153.378 

206.819 

- 104.023 

10 

1 

-1 

-1 

1 


1 

82.765 

97.702 

87.739 

94.618 

- 90.194 

11 

-1 

1 

-1 

1 


1 

128.789 

159.214 

131.805 

154.813 

- 99.436 

12 

1 

1 

-1 

1 


-1 

83.854 

98.019 

109.475 

98.034 

- 91.652 

13 

-1 

-1 

1 

1 


1 

153.453 

216.126 

154.486 

207.743 

- 104.435 

14 

1 

-1 

1 

1 


-1 

83.720 

101.371 

85.523 

99.925 

- 90.649 

15 

-1 

1 

1 

1 


-1 

128.261 

159.511 

130.653 

174.621 

- 100.152 

16 

1 

1 

1 

1 


1 

83.644 

94.659 

110.132 

91.651 

- 91.184 

17 

0 

0 

0 

0 


0 

97.714 

118.274 

100.694 

114.897 

- 93.690 

18 

-1 

0 

0 

0 


0 

137.437 

174.931 

139.790 

170.091 

- 101.060 

19 

1 

0 

0 

0 


0 

83.531 

96.579 

90.889 

93.246 

- 90.258 

20 

0 

-1 

0 

0 


0 

100.726 

124.102 

103.103 

120.556 

- 94.476 

21 

0 

1 

0 

0 


0 

95.844 

114.538 

99.449 

111.058 

- 93.176 

22 

0 

0 

-1 

0 


0 

97.679 

118.160 

100.654 

114.810 

- 93.677 

23 

0 

0 

1 

0 


0 

97.759 

118.347 

100.751 

114.936 

- 93.700 

24 

0 

0 

0 

-1 


0 

104.244 

122.517 

109.246 

117.315 

- 94.645 

25 

0 

0 

0 

1 


0 

96.731 

118.287 

99.177 

115.201 

- 93.600 

26 

0 

0 

0 

0 


-1 

96.856 

118.396 

99.271 

121.655 

- 93.938 

27 

0 

0 

0 

0 


1 

98.500 

118.874 

102.004 

114.817 

- 93.805 
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Table 9: Coefficients for Model of Signal-to-Noise Ratio of System Response 


Coefficient 

Value 

Coefficient 

Value 

Coefficient 

Value 

Coefficient 

Value 

bo 

-93.4849 

b 4 

0.8526 

bn 

0.1848 

b 22 

-0.3668 

bi 

5.3657 

b 5 

0.0723 

b 2 3 

0.2449 

b 3 3 

-0.2291 

b 2 

0.7772 

bi2 

-1.3888 

bis 

-0.1675 

b44 

-0.6631 

b 3 

-0.1351 

bi3 

0.1913 

bn 

-2.2001 

b 5 5 

-0.4123 


6.3. Optimal Parameter Design 

The optimizing compiler in Excel was used to determine the maximum signal-to-noise ration within 
the range of the control factors using the model in the previous section. The optimal signal-to-noise ratio 
was SNoptimai = -89.62 with the parameter values given in Table 10. 


Table 10: Optimal Values of the System Control Factors 


Control 

Factor 

Value 

Control 

Factor 

Value 

Ki 

3.750 

X 

27.826 

Ki 

1.035 

8 

0.016 

Ki 

0.080 



6.4. Verification 

A brief experiment was run to verify the optimization results. For this verification experiment, the 
control factors were set as in Table 10 and noise factors were cycled as in the outer array in Table 8. The 
resulting signal-to-noise ratio was SNverification = -90.082, which is a close match to the optimal value. Figure 
7 to Figure 10 show the arm position tracking response for each verification test point. The steady-state 
oscillations are probably due to time delays in the system and the torque dead zone of the motor. Figure 1 1 
shows the time history of all the plant state variables for the verification run with Noise Factors set to (Ni, 
N 2 ,N 3 ) = (+1,+1,+1). 


15 





Arm Position 



Arm Position 
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Arm Position 
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Figure 10: Verification Run for Noise Factors (Nl, N2, N3) = (+1, +1, +1) 
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Figure 11: Plant State Variables for Verification Run with Noise Factors (Nl, N2, N3) = (+1, +1, +1) 


7. Final Remarks 

This case study has demonstrated the application of Reliability Engineering techniques for optimal 
tuning of a complex control system. These techniques pertain to experiment design and mathematical 
modeling of the effects of control and noise factors on system performance and robustness. The applied 
approach enables the generation of insight into the relation between goals, dependencies, and uncertainties 
in a system. This is especially useful when complexity limits our ability to derive closed-form analytical 
expressions of system performance. The examined system contained many complex features such a dual- 
loop control architecture, plant and controller state (i.e., memory), and a variety of nonlinear relations 
including quadratic, sinusoidal, and thresholds. 

The demonstrated approach is simple, structured, effective, repeatable, and cost and time efficient. This 
general approach is suitable for a wide range of systems. The case study showed that the applied techniques 
are able to predict performance with high accuracy. Simulation-based experimentation in a flexible 
environment such as MATLAB enables quick turnaround in the generation, characterization, and tuning of 
virtual system prototypes. 

There are several ways to expand the scope and depth of the case study presented here. Future case 
studies could increase the number of plant and controller parameters in the sets of control and noise factors. 
Also, instead of a single ramp input signal, a set of input signals could be applied such as steps, square 
waves, and sinusoidal waveforms. In that case the input signals would be handled as additional noise factors 
and considered in the design of the experiment outer array. This would expand the set of conditions in the 
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characterization of robustness. Additionally, in this case study, we chose to follow a trial-and-error 
approach to identify a stable system configuration that served as a baseline for the study of performance 
and robustness. In tuning a nonlinear system from a stability perspective, we are primarily interested in the 
size of the region of convergence. It should be possible to develop simple metrics for the size of the region 
of convergence. Given that, an experiment could be designed to explore the system design space and 
identify the proper tuning of parameters to achieve an optimal balance of stability, performance, and 
robustness. 
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Appendix A. MATLAB Script for Simulation Experiment 


% Run Experiment on pendulum control system 
% Notes: 

% 1. Performance measure results are saved to file 

o, 

o 

echo on ; 


9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-Q-9- 

ooooooooooooooooooooooooooooo 

% Set filename for control and noise matrices 
% => This one in MATLAB format 

% => This one will be in Excel-loadable format 
control_matrix_f ilename_TXT = 'control_matrix.dat' ; 
noise_matrix_f ilename_TXT = 'noise_matrix.dat' ; 

Q,Q,0,0,g,Q,0,Q,Q,Q,0,Q,Q,Q,0,Q,Q,0,0,Q,Q^Q,0,Q,Q,Q,0,Q,Q, 

OOOOOOOOOOOOOOOOOOOOOOOOOOOOO 

% Set filename for error matrix 
% => This one in MATLAB format 

% => This one will be in Excel-loadable format 
E_matrix_f ilename_TXT = ’ E_matrix . dat ' ; 

Pl_matrix_f ilename_TXT = 'Pl_matrix.dat' ; 

Pu_matrix_f ilename_TXT = 'Pu_matrix.dat' ; 
e21_matrix_f ilename_TXT = 'e21_matrix.dat' ; 
e2u_matrix_f ilename_TXT = 'e2u_matrix.dat' ; 

o, 9- 9- 9- 9- 9- 9- 9- S- 9- &- 9- 9- 2- 9- 9- 9- S- 9- S- 9- 2- 9- 5- 9- S- 2- 0- 

ooooooooooooooooooooooooooooo 

% Set filename for performance measures 
% => This one in MATLAB format 

perf_measures_f ilename_MAT = ' perf_measures_MAT ' ; 

Q,Q,0,Q,Q,Q,0,Q,Q,0,0,Q,Q,Q,0,Q,Q,Q,0,Q,Q,Q,0,Q,Q,Q,0,Q,Q, 

OOOOOOOOOOOOOOOOOOOOOOOOOOOOO 

% Set filename for signal-to-noise measures 
% => This one will be in Excel-loadable format 

% ==> Make entries TAB separated and include end-of-line character 
% ==> Can also include headings (for rows, columns, table caption, 
SN measures filename TXT = ’ SN measures.dat' ; 


ooooooooooooooooooooooooooooo 

% Set model information 

ooooooooooooooooooooooooooooo 

% Specify model name 

model_name = ’ system_controller_n_plant_enhanced_v3p0 ’ ; 

% Run time (seconds) 
t_f inal = 30.0 ; 

% Transient interval duration (seconds) 
t Transient = 15.0 ; 


% Set plant initial conditions 

Initial_arm_angle_deg 

Initial_arm_speed_deg_sec 

Initial_pendulum_angle_deg 

Initial_pendulum_speed_deg_sec 


[0 0 . 0 ] / 

[0 0 . 0 ] / 

[0 0 . 0 ] / 

[0 0 . 0 ] / 


etc . 
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Baseline controlled model parameters 


kl base = 

2.50 ; 

o, 

o 

XI 





k2 base = 

2.00 ; 

o, 

o 

X2 





k3 base = 

0.10 ; 

o_ 

o 

X3 





lambda base = 

20.0 ; 

Q. 

O 

X4 





epsilon base = 

0.017 ; 

Q_ 

O 

X5 





eta base = 

1.0 / 

O, 

o 

This one 

will 

. remain constant 

% Baseline noise (uncontrolled) model 

parameters 


J1 kg m2 base 

= 0.04699 

r 

% This 

one 

will 

remain 

constant 

J2 kg m2 base 

= 0.007800 

; % This 

one 

will 

remain 

constant 

LI m base 

00 

o 

II 


% N1 





L2 m base 

00 

o 

II 


% N2 





m kg base 

= 1.1564 

r 

% N3 






% Range percentage for model parameters (%) 
r_control = 50.0 ; 
r_noise = 15.0 ; 

Q-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9-9- 

ooooooooooooooooooooooooooooo 

% Set experiment matrices 

ooooooooooooooooooooooooooooo 

% Matrix for Controlled parameters 
% XI, X2, X3, X4, X4, X5(= X1X2X3X4) 
Control_matrix = ... 

[ 


\ — i 

i 

\ — 1 

1 

\ — 1 

1 

\ 1 

1 

\ 1 

+ 

\ — i 
+ 

-1, 

-1, 

-1, 

-l; 

-i. 

+ 1 1 

-1, 

-1, 

-l; 

+i. 

+ 1 1 

-1, 

-1, 

+1 ; 

\ — i 

i 

-1, 

+1, 

-1, 

-l; 

\ — i 
+ 

-1, 

+1, 

-1, 

+1; 

\ — i 

i 

+ 1 r 

+ 1 1 

-1, 

+1 ; 

\ — i 
+ 

+ 1 1 

+ 1 ! 

-1, 

-1; 

\ — i 

i 

-1, 

-1, 

+ 1, 

-1; 

\ — i 
+ 

-1, 

-1, 

+ 1, 

+1; 

-i. 

+ 1 r 

-1, 

+ 1 r 

+1 ; 

\ — i 
+ 

+ 1 ' 

-1, 

+ 1, 

-1; 

\ — i 

i 

-1, 

+1, 

+ 1, 

+1; 

\ — i 
+ 

-1, 

+1, 

+ 1, 

-l; 

-i. 

+ 1 r 

+ 1 r 

+ 1 r 

-1; 

+i. 

+ 1 ' 

+ 1, 

+ 1 r 

+1; 

0, 

0, 

0, 

Or 

0; 

-1 , 

o, 

0, 

o, 

0; 

+ 1 r 

o, 

0, 

o, 

0; 

0, 

-1, 

0, 

o, 

0; 

0, 

+ 1, 

0, 

Or 

0; 

0, 

0, 

-1, 

Or 

0; 

0, 

0, 

+ 1 r 

o, 

0; 

0, 

0, 

Or 

~lr 

0; 

0, 

0, 

Or 

+ lr 

0; 
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0, 

0, 

o, 

0, 

-1; 

0, 

0, 

0, 

0, 

+1 ] ; 

% Matrix 

for 

Nc 

iise parameters 

% Nl 

, N2 

, N3( 

= 

N1N2 ) 

Noise matrix 

= 

. . . 

t 





-i , 

-1, 

+1 ; 



+i , 

-1, 

-l; 



- 1 , 

+ 1 r 

-l; 



+i i 

+ 1 ' 

+1 ] 

r 



% Performance Measure matrices 


weights 

num_me tries 

num_ctrl_rows 

num_ctrl_columns 

num_noise_rows 

num noise columns 


= [0.25, 0.25, 0.25, 0.25] 
= 4 / 

= size (Control_matrix, 1) 
= size (Control_matrix, 2) 
= size (Noise_matrix, 1) 
= size (Noise matrix, 2) 


PI = zeros (num_ctrl_rows , num_noise_rows ) 
Pu = zeros (num_ctrl_rows , num_noise_rows ) 
e21 = zeros (num_ctrl_rows , num_noise_rows ) 
e2u = zeros (num_ctrl_rows , num_noise_rows ) 
U = zeros (num_ctrl_rows, num_noise_rows , 
E = zeros (num_ctrl_rows , num_noise_rows ) 
SN = zeros (num Ctrl rows, 1) ; 


num metrics) 


spause 


% Loop over Control Matrix 
for i = 1 : num_ctrl_rows 

% Factor values for current control point 


kl 


k2 


k3 


lambda = 


[0 

[0 

[0 

[0 


epsilon = [0 


eta 


(kl_base * ... 

(1 + Control_matrix ( i , 1 ) _ 

(k2_base * ... 

(1 + Control_matrix ( i , 2 ) *r_control/100 . 0 ) 
(k3_base * ... 

(1 + Control_matrix ( i , 3 ) *r_control/100 . 0 ) 
(lambda_base * ... 

(1 + Control_matrix ( i , 4 ) *r_control/100 . 0 ) 
(epsilon_base * ... 

(1 + Control_matrix (i, 5) *r_control/100 . 0) 
[0 eta base] ; % This one remains constant 


h control/100.0)) ] 


) ] 


) ] 


% Loop over Noise matrix 
for j = 1 : num_noise_rows 

% Factor values for current noise point 


This one remains constant 
This one remains constant 


Jl_kg_m2 = [0 Jl_kg_m2_base ] ; 

J2_kg_m2 = [0 J2_kg_m2_base ] ; 

Ll_m = [0 (Ll_m_base * (1 + 

Noise_matrix ( j , 1) 

L2_m = [0 (L2_m_base * (1 + 

Noise_matrix ( j , 2 ) *r_noise/100 . 0 ) ) ] 


^r noise/100 . 0 ) ) ] 
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m_kg 


= [0 (m_kg_base * (1 + 

Noise_matrix ( j , 3) *r_noise/100 . 0) ) ] ; 

^g,Q.^Q.Q,Q.Q.O,Q,Q.Q.g.g,g.Q.0,Q,Q,Q.Q.Q,g.Q.0,g,0,Q,0, 

ooooooooooooooooooooooooooooo 

% Run the model 

plant_state = sim(model_name, ' StopTime', , t_final') ; 

ooooooooooooooooooooooooooooo 

% Get the plant state histories 

time = plant_state . get (' time ’ ) ; 

arm_position_target_deg = 

plant_state . get ( ’ arm_position_target_deg ’ ) ; 
arm_position_deg = plant_state . get ( ' arm_position_deg ' ) 

arm_speed_deg_sec = plant_state . get ( ’ a r m_sp e e d_de g_s e c ' ) 

pendulum_position_deg = 

plant_state . get ( ' pendulum_position_deg ' ) ; 

pendulum_speed_deg_sec = 

plant_state . get ( ' pendulum_speed_deg_sec ' ) ; 

%show_plant_state_vl 


pause 


% Get number of time points in the simulation 
num_time_points = length (time) ; 

% Scan full length of simulation run 
for k = 1 : num_time_points 

% For Lower/Transient time interval 
if (time(k) <= t_Transient) 

Pl(i, j) = PI (if j ) + • • • 

(arm_position_deg ( k) - ... 

arm_position_target_deg ( k) ) A 2 ; 
e21(i,j) = max ( [e21 ( i , j ) , ... 

(arm_position_deg ( k) - 
arm_position_target_deg ( k) ) A 2 ] ) ; 

% For Upper/Steady-state time interval 
else 

Pu(i, j) = Pu(i f j) + . . . 

(arm_position_deg ( k) - 

arm_position_target_deg ( k) ) A 2 ; 
e2u(i f j) = max ( [e2u ( i , j ) , ... 

(arm_position_deg ( k) - 

arm_position_target_deg ( k) ) A 2 ] ) ; 

end 

end 


% Finish up averaging sums of squares 
Pl(i,j) = Pi ( i , j ) /num_time_points ; 
Pu(i,j) = Pu ( i , j ) /num_time_points ; 


% Store basic measures 
U (i, j , 1) = Pi (i, j ) ; 

U (i, j , 2) = e21 (i, j ) ; 

U ( i , j , 3 ) = Pu ( i , j ) ; 
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U (i, j , 4) = e2u (i, j ) ; 


% Computer composite measure 
for k = 1 : num_me tries 

E(i, j) = E(i,j) + weights (k) *U (i, j, k) ; 
end % k 

% Update signal-to-noise ratio with current composite measure 
SN ( i ) = SN(i) + E(i, j)*E(i, j) / 


%pause 

end % j 

% Compute Signal-to-Noise ratio for current control point 
SN(i) = -10*log (SN (i) /num_noise_rows ) ; 

%pause 

end % i 

ooooooooooooooooooooooooooooo 

% Save experiment control and noise matrices in Excel-loadable format 
save ( control_matrix_f ilename_TXT, ' Control_matrix ' , ' -ascii ' , ' -tabs 1 ) 

save (noise_matrix_f ilename_TXT, ' Noise_matrix ' , ’-ascii', ’-tabs’) 

0^Q,Q,Q_O.Q_Q_Q_CuQ_Q,Q_Q,Q_Q,Q_Q,OQ,guQ_Q_Q,Q_Q_Q_Q.Q_Q_ 

OOOOOOOOOOOOOOOOOOOOOOOOOOOOO 


% Save performance matrices in MATLAB format 
save (perf_measures_f ilename_MAT, . . . 


'PI', ' e21 ' , 'Pu', ' e2u ' , ’ 

'U' , 'E 

', 'SN') 

r 



ooooooooooooooooooooooooooooo 

% Save performance matrices in 

Excel- 

loadable 

format 


save (PI matrix filename TXT, 

'Pl' , 

' -ascii ' 

i 

r 

' -tabs ' ) 

r 

save (Pu matrix filename TXT, 

'Pu' , 

' -ascii ' 

i 

r 

' -tabs ' ) 

r 

save (e21 matrix filename TXT, 

' e21 ' , 

' -ascii ' 

\ 

r 

' -tabs ' ) 

r 

save (e2u matrix filename TXT, 

' e2u ' , 

' -ascii ' 

i 

r 

' -tabs ' ) 

r 

save (E matrix filename TXT, 

'E', 

' -ascii ' 

i 

i 

' -tabs ' ) 

r 

save (SN measures filename TXT, 

GO 

3 

' -ascii ' 

\ 

r 

' -tabs ' ) 

r 


ooooooooooooooooooooooooooooo 

% Finished 
echo off ; 
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